Feature Selection using Eigenvalue Optimization and Partial Least Squares

Feature selection is an essential problem in many fields such as computer vision. In this paper we introduce a supervised feature selection criterion based on Partial Least Squares regression (PLS). We find an optimal feature subset by applying the theory of Optimal Experiment Design to optimize the eigenvalues of the loadings matrix obtained from PLS. Since PLS extracts components such that the covariance between features and response variable and the covariance between features itself are simultaneously maximized, the criterion simultaneously satisfies the relevance property towards the response variable and the latent information in features. In order to optimize the eigenvalues, we use the D-optimality criterion which maximizes the determinant of loadings covariance matrix. The paper first introduces a theoretical proof of the proposed criterion followed by empirical evaluation using two image data sets. Our experiments demonstrate that our Optimal Loadings criterion outperforms other popular supervised feature selection techniques.